Skip to content

Conversation

mzeitlin11
Copy link
Member

For example, on master we generate:

@cython.wraparound(False)
@cython.boundscheck(False)
def take_2d_axis0_int8_int32(const int8_t[:, :] values,
                                    ndarray[intp_t] indexer,
                                    int32_t[:, :] out,
                                    fill_value=np.nan):
    cdef:
        Py_ssize_t i, j, k, n, idx
        int32_t fv

    n = len(indexer)
    k = values.shape[1]

    fv = fill_value

    IF False:
        cdef:
            const int32_t *v
            int32_t *o

        # GH#3130
        if (values.strides[1] == out.strides[1] and
            values.strides[1] == sizeof(int32_t) and
            sizeof(int32_t) * n >= 256):

            for i in range(n):
                idx = indexer[i]
                if idx == -1:
                    for j in range(k):
                        out[i, j] = fv
                else:
                    v = &values[idx, 0]
                    o = &out[i, 0]
                    memmove(o, v, <size_t>(sizeof(int32_t) * k))
            return

    for i in range(n):
        idx = indexer[i]
        if idx == -1:
            for j in range(k):
                out[i, j] = fv
        else:
            for j in range(k):
                out[i, j] = values[idx, j]

On this pr:

@cython.wraparound(False)
@cython.boundscheck(False)
def take_2d_axis0_int8_int32(const int8_t[:, :] values,
                                    ndarray[intp_t] indexer,
                                    int32_t[:, :] out,
                                    fill_value=np.nan):
    cdef:
        Py_ssize_t i, j, k, n, idx
        int32_t fv

    n = len(indexer)
    k = values.shape[1]

    fv = fill_value


    for i in range(n):
        idx = indexer[i]
        if idx == -1:
            for j in range(k):
                out[i, j] = fv
        else:
            for j in range(k):
                out[i, j] = values[idx, j]

Not too impactful since just generated code, but makes looking at all the take helpers clearer (and the file shorter).

@mzeitlin11 mzeitlin11 added Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff Clean labels Jun 26, 2021
Copy link
Member

@jbrockmendel jbrockmendel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jreback jreback added this to the 1.4 milestone Jul 1, 2021
@jreback jreback merged commit 5d15520 into pandas-dev:master Jul 1, 2021
@jreback
Copy link
Contributor

jreback commented Jul 1, 2021

thanks @mzeitlin11

@mzeitlin11 mzeitlin11 deleted the cln_take_template branch July 1, 2021 23:45
JulianWgs pushed a commit to JulianWgs/pandas that referenced this pull request Jul 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff Clean

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants